Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SNOW-1649780: Bug fix Series.sort_values fails when name overlaps with index name #2291

Merged
merged 1 commit into from
Sep 18, 2024

Conversation

sfc-gh-nkumar
Copy link
Contributor

@sfc-gh-nkumar sfc-gh-nkumar commented Sep 13, 2024

In native pandas, calling sort_values where column name overlaps with index name raises error ValueError: 'col' is both an index level and a column label, which is ambiguous. But this works for series.

In Snowpark pandas we were raising errors both the scenarios Dataframe (this is expected) and Series (this was bug).
Root cause: In modin frontend we to implement Series.sort_values we convert this to Dataframe and call sort_values on it. So this failed the validation above.
Fix: Instead of converting Series to Dataframe directly call query_compiler.sort_rows_by_column_values.

@sfc-gh-nkumar sfc-gh-nkumar requested a review from a team as a code owner September 13, 2024 19:04
@sfc-gh-nkumar sfc-gh-nkumar force-pushed the nkumar-SNOW-1649780-sort-bug branch from f2f19b5 to 84aab65 Compare September 13, 2024 19:05
@sfc-gh-nkumar sfc-gh-nkumar added the NO-PANDAS-CHANGEDOC-UPDATES This PR does not update Snowpark pandas docs label Sep 13, 2024
@sfc-gh-nkumar sfc-gh-nkumar force-pushed the nkumar-SNOW-1649780-sort-bug branch 2 times, most recently from a70a024 to dcb7531 Compare September 16, 2024 17:46
Copy link
Contributor

@sfc-gh-rdurrani sfc-gh-rdurrani left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left a minor nit, but lgtm otherwise!

@sfc-gh-nkumar sfc-gh-nkumar force-pushed the nkumar-SNOW-1649780-sort-bug branch 2 times, most recently from dd0bbee to 940915e Compare September 16, 2024 23:14
@sfc-gh-nkumar sfc-gh-nkumar force-pushed the nkumar-SNOW-1649780-sort-bug branch from 940915e to 0f54961 Compare September 17, 2024 05:28
Copy link
Contributor

@sfc-gh-rdurrani sfc-gh-rdurrani left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@sfc-gh-nkumar sfc-gh-nkumar force-pushed the nkumar-SNOW-1649780-sort-bug branch from 0f54961 to c5ab043 Compare September 17, 2024 20:43
@sfc-gh-nkumar sfc-gh-nkumar force-pushed the nkumar-SNOW-1649780-sort-bug branch from c5ab043 to 12b64cb Compare September 18, 2024 18:38
@sfc-gh-nkumar sfc-gh-nkumar merged commit 6841488 into main Sep 18, 2024
35 checks passed
@sfc-gh-nkumar sfc-gh-nkumar deleted the nkumar-SNOW-1649780-sort-bug branch September 18, 2024 21:10
@github-actions github-actions bot locked and limited conversation to collaborators Sep 18, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
NO-PANDAS-CHANGEDOC-UPDATES This PR does not update Snowpark pandas docs snowpark-pandas
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants